


ae 





“Calhoun 


Institutional Archive of the Naval Postgraduate School 


Calhoun: The NPS Institutional Archive 
DSpace Repository 


Theses and Dissertations 1. Thesis and Dissertation Collection, all items 


1968-06 


An algorithm for the solution of linear 
programming problems. 


Sparks, Donald Leroy 


Monterey, California. Naval Postgraduate School 


http://ndl.handle.net/10945/11575 


Downloaded from NPS Archive: Calhoun 


| Calhoun is the Naval Postgraduate School's public access digital repository for 
D U DLEY research materials and institutional publications created by the NPS community. 
get Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS'‘s first 
KNOX appointed — and published — scholarly author. 


LIBRARY Dudley Knox Library / Naval Postgraduate School 
411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 





http://www.nps.edu/library 


NPS ARCHIVE 


1968 
SPARKS, D. 





AN ALGORITHM FOR THE SO 
’ fAIE. LUTIC t 
LINEAR PROGRAMING PROBLEMS ai 


DONALD LEROY SPARKS 











AN ALGORITHM FOR THE SOLUTION OF 
LINEAR PROGRAMMING PROBLEMS 
by 
Donald Leroy Sparks 


Captain, United States Army 
B.S., Oklahoma State University, 1963 


Submitted in partial fulfillment of the 
requirements for the degree of 


MASTER OF SCIENCE IN OPERATIONS RESEARCH 
from the 


NAVAL POSTGRADUATE SCHOOL 
June 1968 


ABSTRACT 


Linear programming techniques are becoming of greater 
importance because the use of computerization has increased 
the fields for applications for linear programs. The primal- 
dual algorithm, in which the constraints are added one at a 
time, is investigated as a possible faster solution method. 

A computer program was developed to compare this method with 
the standard primal-dual algorithm using the full set of 
constraints at one time. Several :random problems were solved 
using these two methods, and the results indicated a signif- 
icant improvement in the solution time by the use of adding 


the constraints one “at a time. 
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lo = tN LRODUeT Ter 

New linear programming algorithms have been developed to 
reduce the computational time in solving linear programs. The 
purpose of this thesis is to investigate the merits of one 
such new algorithm. This method consists of introducing the 
constraint equations one at a time. After each constraint is 
added, the "smaller", or submatrix, problem is solved using 
the primal-dual algorithm. This continues until all con- 
Straints have been added and a solution is obtained. 

The rationale for this approach is that small matrices 
are used in the initial stages of solving the linear program; 
the size of the matrices increases only when additional con- 
straints are introduced. If the number of iterations used in 
this method is not significantly different from the number of 
iterations used with the full matrices, the manipulation of 
the smaller matrices in the initial stages will reduce the 


SoelutLom ECilmes 


al 


LivweeeNOGTATION 
number of constraint equations. 
number of legitimate variables. 
mxn matric of coefficients of the constraint equations 
with elements ee ed eee | es eel Re 
mxm matrix of the basis vectors. 


inverse of the basis. 


mxl column vector which is the yeh column of A. 

Byes 

15 

1. .Cuie, We = ° , ae = ae, am « 
L 

mxl column vector associated with the i” artificial 
Varlable, 1 =se...;m . 
mxl column vector with elements a 4 6 where = = po'P,. 


nxl column vector with elements Bhs which are the costs 
of the legitimate variables. 

mxl column vector with elements b, which are the right- 
hand sides of the constraint equations. 


dual slack variables. 
dual slack variables after a dual iteration. 


mxl column vector whose elements are the coefficients 
of the basis variables of the added constraint 


equation. 


Notation used with tableau: 
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d; the vector to be introduced into the basis. e.g. 


in this tableau Po Will be introduced. 


(2) pivot element for a primal iteration, i.e., the 


6 Giaeterion 2s 6 = mMin(x../x.#) sche ehacex.. = 
Be. 1) 
pivot element for a dual iteration, l1.e., the 


6-criterion is 6 = min (-s./2.-¢.) such that 
2, 
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Ill. FORMULATION OF THE PROBLEM AND SOLUTION PROCEDURE 


The general linear programming problem is to maximize 


n 
Z = ) Cage. 
j=1 Dieta) 
subject to 
: | 
ARK OS eee ae. ee (1) 
a= | nl ey 1- 
and 
Suc ee () | ee as ee 


J 


The modified pramal wses an additional constramme 


where the cost of Xo 1s zero and Dy is arbitrarily large, so 
that for x) > 0 the constraint adds no additional restriction 
ong) ; 


The modified primal is written to maximize 


} 
7 Cex 
4=1 ees) 
subject to 
} 
X, + Xo ID. . 
0 al q 0 
(2) 
n 
) aS = Op ds el, 7M , 
4=1 ag) a 
and 
xX. > 0, J =1,-.-,n . 


From (2) we can write the modified dual with slack var- 


iables, S50 j = 6,1,...,n,"added. That 15 °to minimise 


subject to 


Wo - S) = 0, 
m 
Wo + ALY W i - 24 = rv | =e oe (3) 
and 
Ws Mmanestricted= for 1 =. 0 25) 
The starting feasible solution to the primal dual al- 
gorithm is va Os i ae oe Wo = Max ( cue)... 


y 
For an optimal solution the complementary slackness con- 


Grecion must Tela. That is 
n 
ee ee eee (4) 


Adding artificial variables, Kogs 1 =)0 hae. y tome 
with the cost of the artificial variables set to -l and the 
cost of the legitimate variables set to zero, the extended 


primal Can De written as 


maximize 
m 
ate a Xai 
subject to 
} 
x, + -. +X ae 
0 sat 5 a0 0 
} 
x. + x = ee Ie aa ae. | My 
and 
fo Cd] ye a ne ary XG 2 0, 1 = 0, ,m 


Solving the extended primal is similar to using a Phase 


(1) 


I Revised Simplex method. However, in the primal-dual 
algorithm, when Phase I ends the linear program is solved 
because complementary slackness is maintained throughout the 
solution procedure. 

If a new constraint is added to the tableau, complemen- 
tary slackness is maintained without changing the dual slack 
variables. This can be shown as follows: 

Assume we have a feasible solution to the problem 

with k constraint equations. This means that S. = 0 

for all j such that Bs € P and ‘ee = 0 for all j such 

that Pe ¢€ P. These conditions imply that complemen- 
tary slackness is maintained, and that the modified 
dual also has a feasible solution. 

Now we add the k + ae constraint which introduces 

a new dual variable, Weaqe but no new dual slack 

variables, Sa, j = 0,l,...,n. We need a feasible 

solution to the modified dual for the enlarged 

system. Observe that we have a feasible solution if 

we set Wide s 0 since then the So, jj = 0p. she 

remain unchanged. In particular, ae =,0 £6r ali, 

such that ae € P, that is, for the legitimate var- 

jabies. Alsey x. = 0 for all jesmeh that P. ¢ p, 

which implies that we have maintained complementary 

slackness. 

It 1s worth noting that the Z4—Cas 7 = 0,1,..-.@n, must be 


recalculated since the new constraint which is added to the 
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extended primal starts with its artificial variable, a ee 
a 

in the basis with 1ts eect cect sone. 

Pe 6 0 
The new basis is P = # , where © arti = 

d It I 

is the artificial vector associated with Now we can 

a 


solve the new k + 1 system using the primal-dual algorithm 
since complementary slackness has been maintained. 
An optimal solution exists if and only if the following 


criteria are satisfied: 


Loe Mee 8 a0) Stor oy 0 ... apm, 
Coie Nees 

Ze Zp-CR = 0, and 

oH Xo cae |e 


The solution procedure is as follows: 

The first tableau 1s set up using the first two 
constraints of the extended primal and the starting 
solution to the modified dual, which implies that 
at least one = = 0. 


Step 1. Is there a j, Say Joe such that Sag = 0 and 


an 6 eS Goto. 2 . 

Db. =Nemmn Ge. tous. 
Step 2. Imeroduce 0 into the basis using the minimum 
6-criterion and a primal iteration. Since the extended 
primal is bounded a pivot will always exist. Note that the 
3 remain unchanged for all j. Gotol. 
Step 3. Is aes < 0 for some j? 

a. Yes. Go to 4. 


Dea NOem EGO FO 5. 
Jak 


wn 


step 4. Use the minimum O0-criterion. Is 86 bounded? 
a. Yes. Perform a dual iteration to compute a new 
set of ee say s.'S. Gomeo. 1. 


b. No. The linear program has no feasible solution. 


= 2 
Step Se Is Zn Cp < 07 


a. Yes. The linear program has no feasible solution. 


D. Nom aGemae.o. 
Step 6. Have all of the constraints been added? 

a. ees. Gomeo 2. 

b.. Ne. Gomto 7. 
Step 7. Introduce the next restraint, say the k + li 
Place the artificial vector P in the basis. Compute 


a,k+l 


Is =O. 


“B,k+1° 7B) ee 

a. gies. GOVteme, 

b. No. Multiply all coefficients of the new constraime 
except for the artificial variable, by -l. This assures that 
XB k+l 2 0 and the artificial variable is non-jegative> wee 
EO 16e 
Step 8. For thessystem with k + 1 rYestraints, compute “cae 
new values of age FORMe = Oj, ..ni7n) GOmre ee. 

Step 9. Is Xo = 0? 


a. Yes. The linear program is unbounded. Stop. 


b. No. An optimal solution has been found. Stop. 


me 
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SAMPLE PROBLEM 


Consider the following example: 


maximize 


—— 2X) 
subjectere 
3X4 + 4x. + 6x. 
4x, + 3X5 + 12x, 
xy + Xo + 4x4 
and 
> re ae 0 eas 
i J 
Then the extended primal is 
maximize 
‘ *50 q al “ 
subject to 
Xo + Xy ~ Xo + X 3 = Xy + x 
3X4 + 4X. = OX. + Xy 
ax, + 3X5 a 12x, + x 
xy + X5 aa 4x. 
Ba > 0 Bee, = 0, &. . Gay 


The dual slack variables are 


Then Sp 


that s 


ih 


24 


24 


il 


The starting tableau, using the first original constraint, 


is 





From step 1, we see that S3 = 0 and Z,-C, < 0. Using 


the minimum @-criterion (as discussed in section III) in step 


2, we introduce P into the basis and remove Ry from the 


iL 


basis. 


Since there is no Jo for which =a = rane Ay < "OF 


evn 
bulk Bs Ea < 0 for several values of j, we arrive at step 4. 
Using the minimum 6-criterion a new set of s.'s, called S.'s 


are calculated. 





Now S_= 0 and Zy~o < 0 so, from step 2, we introduce 


Po into the basis and remove P 0 from the basis. 
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From the above tableau we trace through steps lb, 3b, 
5b, 6b, and arrive at step 7. Note that we have obtained an 
optimal solution to the subproblem with one constraint. In 
step 7 we introduce the second constraint. Since P, and P 


0 3 


are basis vectors, at = (A509 7A5 3) = (0,12); the new basis 


consists of Por P. and Pio With this basis we find that 
Xpo = -24 < 0 so that step /b must be used. The second re- 


straint is replaced by 


5 + Xoo = -24 , 


which is used throughout the remainder of the solution proce- 


dure. Note that now at = (0 r= 2 mang Xpo = 24 > 0. New 


values of aa for j = 0,1,...,;nm are computed (step 8), and 


the new tableau 1s 








i i me) 


OD 2 


Bance a) 0 and Z4-C, < 0 (step 1) we go to step 2. 


Using a primal iteration, we introduce Py into the basis and 


eliminate P. £voOmuehiec baste. 


aac ee 
wBIrT73 = 3c 
A ee 
6 = =e l72Z 





Now Barc gu0 for gad) ) efoxg@h ich S, = Ow From steps ae 
3a, and 4a, a new set of aes are computed. Then Sy becomes 
zero and Z,-C, < 0 so, from step De, P, enters the basis and 


P25 is removed from the basis. 


al a2 
=) “=z 
O -1/4 
1 3/4 





Optimality has now been obtained with the second restraint 


added. Following steps lb, 3b, 5b, 6b, and 7, we introduce 
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the third and@ final restraint. “Simee the basis vectors were 


=—T : : 
Pos Pie and Py, a = (839183) 1434) ==( 6-10). MPhe newesadsus 
vectors are Pos Po, Par P 3° We find that Xp 3 = 2 > 0 so we 
go to step 8 and recompute Bao rOr | = "eee... 7 n 


The next sequence of steps is lb, 3a, and 4a, which 


leads to a new set of s.'s. 









0 
Ba OO 1 A 
By iia / hE) 3/7 
ae 1/4 @) 0-1/4 1/4 1 6=2 

z.-c.| -2 --- 6 = -2/-1 

a: 3 / 
Ss. 12 ae 

a) 
S| g it 

J 


Steps la and 2 bring P, into the basis with the elimina- 


jon Of P 3 from the baSisc. The mew talemwean 1s: 





This tableau is the final tableau since the sequence of 
steps lb, 3b, 5b, 6a, and 9b inform us that we have found an 
optimal solution to the original linear program. Note that 


the complementary slackness condition has been maintained, 


that 1S, SX + Jy sors = 0. Therefore, the optimal solution 
is 

Xo = Do = 4 >a0, 

Xy = X5 = Xe = 0, 

X3 = 2, and 

Xy an ae 
with the optimal cost Zz = Sp = B) 
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V. PROGRAMMING TECHNIQUE 

The linear programming technique described in this thesis 
was programmed in FORTRAN IV for use on the IBM 360/67 com- 
puter. One subroutine is used for the primal simplex itera- 
tion and another is used for the dual iteration. The final 
subroutine is used for the addition of constraints. The main 
(driving) routine is used to solve both the full linear pro- 
gram and the linear program using addition of constraints as 
described in this thesis. Using the main routine to solve 
both problems eliminates any time differences due to differ- 
ences in programming techniques. Both of the solution proce- 
dures were timed*, and the number of iterations of each were 
counted. Read and print times were not included in the 


EeminEn? . 


*The timing routine was developed by Lt. E.A. Singer, a 
student at the Naval Postgraduate School. 


oe] 


VI. EFFICIENCY OF THE ALGORITHM 
To eliminate the considerable time and effort required 
to input data by hand, a subroutine was designed which gen- 
erates random problems of a large size. This routine uses a 
random number generator to produce the elements of the A, B, 
and C matrices. The following criteria were used in order 


to insure the existence of a bounded feasible solution: 


maximize 
} 
, ax 
j=1 Sar 
subject to | 
n 
) a. > X oy = Ds, pees eer 
j=1 13 J 
and 


LO me =e wey. ): Sule pene pihls, 
where Xo is the slack variable for the ae constraint. 
For this investigation the subroutine generated problems 
having 70 variables (including 20 slack variables), 20 con- 
straint equations, and 50 cost coefficients using the follow- 


ing uniform distributions: 


ass, Unbeonin sO: 
iy 


b., unieerm (05) 4 


E unlf£orm: (=Le0pe 


aie 
A total of 40 problems were solved using the random 


problem generator described above. The execution times for 


these problems are given in Table I at the end of this section. 
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The comparison of solution times shows that all 40 prob- 
lems ran faster using the method described in this thesis, 
than with the standard primal-dual algorithm. The time dif- 
ferences range from 15.17 seconds to 48.18 seconds with an 


average time difference of 27.53 seconds. 


a 





Prob Full Arra Constraints 
No. Time (x. ) ipietenee Time (y.) Iter Bie 
1 (sec.)* (sec.)* : 
al B72 I SOeoe 21 Gar, 32 
2 87.84 28 569. 22 307.98 
3 80.22 2: 56,50 21 23.69 
4 O73 23 S6en7.0 22 31.23 
5 84.06 22 Syel etal 23 25.45 
6 91.63 24 56262 22 35.00 
7 95.45 25 50)..65 23 36.80 
8 BOs 21 56S 2 2a 23.72 
9 99.60 26 Soo 32 33.28 
10 80.16 21 Syioye ol ai 23565 
11 80.18 2 56.50 Zul: 23.62 
12 ol. 65 24 65rlZ 29 26.58 
iS 80.19 IM oe Sie, 22 22220 
14 91.74 24 64.35 30 27538 
PS 310 20 ail 56.48 21 23372 
16 84.02 22 Sy are 8. 22 26.92 
clky, OS Di2 Pgs, Syc]F S).6 24 36.16 
18 83.95 22 are fill 22 25.24 
19 84.14 22 65.47 25 18uGF 
20 oor 5 26 70.03 28 29.22 
2m 107.08 28 58.90 25 48.18 
22 oy acs 23 Soro Zi 31la324 
23 S470). 22 615339 24 22.62 
24 BORIS Zi SiG aoc aul 23.64 
25 80.17 Za 56.49 yr Ak 23. Ge 
26 80.16 Pah 56. OW Fish 23.66 
27 80.19 2A 5:6...52 Dall 23.68 
28 64.21 22 58.84 Ze 25.38 
29 103... 27 27 56.35 24 45.42 
30 87.83 23 Seat 23 23. 3G 
31 84.07 22 one coe. 22 24.74 
a2 oe. 19 23 56.92 23 31a 
66 80.28 2a 5:6 256 21 23.72 
34 87.96 23 SO noe 2) 31.48 
35 84.11 22 57. OD 23 27.GzZ 
36 84.05 22 50, 10 an 27.49 
cw) 80.20 Px 56. om 21 23.68 
38 S026 21 56.54 20 23.64 
39 oa 3 22 68.86 26 1S. i 
40 o0n 29 Za 56.54 21 23278 
Total 3454.01 2302517 1101.24 
Average CO" SO DIG wroe Zia 


TABLE aL 
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ARIE SUMMARY AND CONCLUSIONS 

A modification of the primal-dual algorithm has been 
presented. This modification differs from the standard 
primal-dual algorithm in that the constraint equations are 
introduced one at a time, and each subproblem is solved be- 
fore the next constraint is added. 

This algorithm was programmed in FORTRAN IV for the IBM 
360/67. The program was designed so that any given linear 
program is solved first by the standard primal-dual algor- 
ithm, and then is resolved using the modified primal-dual 
procedure. Further, the same subroutines are used for both 
methods in order to eliminate timing bias due to coding 
differences. In fact the modified procedure contains steps 
which are not included in the timing of the standard routine. 

In all cases the modified procedure was faster than the 
standard procedure. Of 40 test problems the standard method 
averaged 86.35 seconds per problem, whereas the modified 
method averaged 58.82 seconds per problem. One should not 
judge the actual running time of the test problems since no 
attempt was made to improve the efficiency of the computer 
program on an absolute basis; only the relative speeds of 


the two methods is of importance. 
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| Aes 7 | COMPLETE 
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— TAOGE Rt! 
= | LNrzvIALle@eé | 








Sy 
CALL 
\SIMPLX (j,) “a 
60 
i ade £0 A, x) 
/ 90 IUNB=-1 7 é . : 
=O F Zr Fi-C/ <O _ 
Pe, O POUNDED 


NoTé: 
(1) Iun8= Oj FEASIBLE SOL LITLON 


ExrsTsS (ConTINLE ), 

(2) funG=/ 5 UGapuvded SOLLITT IN 
EXISTS , GUT IRE CONSTRAINTS 
7¢ 8E ADDFES: 

(3) LunyB? 25 UnopundEd SPLLTZ AN 


EXISTS ANDO ALL CONSTRAINTS 
HAVE GEEN ADDED. 
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DUAL ITERATICN SUBROUTINE 


( Dua (yf) 


| COMPUTE 
nae sy S, 





4, 


| CyemrutTe “A 
AND 







nd 
Xia = -d Kg + & 
WHERE ek ok M4 





et are Lage "kia 
BNO MULTI “LY 
RESTRAINT & 
—/ BXCEPT 


BRIT FICL FL, 
VA RIM CLE 





eo 





FORTRAI 


N 





LISTING OF THE COMPUTER PROGRAM 
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